New -O4 tree-level optimization passes, x86 peepholes, and supporting fixes#13
New -O4 tree-level optimization passes, x86 peepholes, and supporting fixes#13joaopauloschuler wants to merge 287 commits into
Conversation
`SizeOf(record.field)` and `BitSizeOf(record.field)` now return the slot the field actually occupies in the surrounding record when a `size N` override is in effect, matching what `OffsetOf` of the next field implies. before, both fell back to the declared type's natural size while the slot was silently padded under them - reading either intrinsic gave a value that did not match `OffsetOf` deltas. mirrors the existing `bitsize N` handling (`BitSizeOf` already honoured `custom_bitsize`). pure type queries (`SizeOf(TypeName)`, `SizeOf(EnumConstant)`) are unchanged - they have no field context.
…ngword `parsed_custom_align` / `parsed_custom_bitalign` in pdecvar.pas and the matching `custom_align` / `custom_bitalign` fields on tfieldvarsym + tfield_sizing_entry change from shortint to longword. PPU read/write switches from byte to longint to match. `alignrecord` parameter and the varalign / varalignrecord locales in symtable.pas widen to longint so they stop wrapping at 128. parse_modern_union's `target_align` widens to longword and gets a high(shortint) cap before assigning into the recordalignment field (which stays shortint - stock FPC infrastructure). Field-level `align N` now accepts powers of two up to 128, 256, 1024 etc. without the shortint wraparound that previously corrupted the sign bit and tripped the `<1` validation. The shortint recordalignment field still caps record-level alignment at 127, so `union align 128` / `record align 128` clamp at 127 rather than overflow. Lifting that cap is a separate change to the stock FPC recordalignment PPU format.
`ncipollo/release-action@v1` with `allowUpdates: true` plus `richardsimko/update-tag@v1` worked only when nothing touched the release between runs. Manually deleting the release, renaming its tag, or moving artifacts left the action in a state it couldn't recover from and the next run failed instead of recreating. `gh release delete --cleanup-tag` followed by `gh release create` is state-independent: whatever the upstream looks like, the run lands at a predictable result. Also resets `published_at` on every build so the release no longer shows a stale "released N days ago" timestamp.
…n enum field An enum-typed record field can only carry the `of T` storage-type modifier, not `size N`. The compiler now rejects the combination with a dedicated error message. Drop the override from `TE.kind` so the test exercises only the legal `of Byte` form, and adjust the matching halts for the natural 1-byte slot size.
equal_signature in pparautl wants te_exact for both params and returndef to bind a method implementation to its forward declaration. The two `(Integer, Integer)` parses produce separate recorddefs that only matched as te_equal, so `function tt.GetPair: (Integer, Integer)` failed to bind to its in-class declaration and the compiler reported the method as a new overload with the same parameter list.
Forward and impl both with anonymous return (`function f: array of string;`) create distinct tdef instances. Pointer-equality compare rejected the impl. Now equal_signature re-points the impl's returndef + hidden funcret paravarsym at the forward's def when both sides have no typesym and are structurally equal.
composable_records PPUs carry recorddef sections with nested enumdefs; loading them in non-composable mode hit insertdef with defowner still nil, since trecorddef.ppuload assigns defowner only after the inner symtable finishes loading.
Adds `m_implicit_generics` to `unleashedmodeswitches`. Existing tests that used the explicit `generic` / `specialize` keywords (which the parser only consumes when implicitgenerics is OFF) drop those prefixes.
Per-unit `{$modeswitch striprtti}` and the `-Mstriprtti` mode-prefix
both reset on any `{$mode X}` directive that follows them. Add a
dedicated global CLI flag `--striprtti` stored as
`force_striprtti_cli` in globals; `rtti_string` short-circuits when
either the modeswitch is active OR the global flag is on. Existing
whitelisting (`expose`, `{$rttiexpose}`, `--rttiexpose=`) keeps
working since the early-return check stays before the whitelist
exits.
`[(1)/100]` failed with `"]" expected but "/" found` because the tuple-aware path in `factor_read_set` consumed `(expr)` itself and swallowed the closing `)`, leaving no one to parse the `/100` tail. factor's own `_LKLAMMER` handler already detects tuple literals at the right nesting level - just delegate to `comp_expr`.
multi-dim and mixing with ranges come free via the comma loop: array[3, 'a'..'z'] becomes array[0..2, 'a'..'z']. N must be a positive integer constant.
Switches the bullet list to a 4-column table covering everything in extra-improvements.md - including compound `+=` and `inc` / `dec` on properties, which the bullets had skipped - and adds the `array[N]` shorthand to the docs/README.md catch-all paragraph.
capture-detection and the capturer-chain skipped blocksymtable, so a closure read the var on its own frame as garbage and outer writes never reached the capturer field. blocksymtable now anchors defowner and symtablelevel at the enclosing procdef rather than the immediate parent (which may be a with-record fieldset or `on E` exceptsymtable whose levels and defowners are unrelated).
Pulling in SysUtils just for its Swap<T>/Exchange<T> generics drags in
exception and handler setup; SwapValues is a System-only builtin. Gated on
{$mode unleashed} and only when no SwapValues symbol is in scope, so it
cannot shadow existing code.
after an anonymous function is reparented into a capturer method (normal_function_level) its block symtables keep the nested level they were anchored to at parse time. a managed inline var in a nested begin..end then had its implicit finalization treated as a parent-frame access: the blocksymtable load saw symtablelevel <> current level and called set_needs_parentfp on a level-2 method, giving IE 2020050302. unmanaged block vars have no synthesized finalization load, which is why only managed types tripped it. guard the parentfp branch on defowner identity: a block var whose blocksymtable belongs to the current procdef lives on the current frame, so skip the parent-fp load. genuine captures (defowner is an outer proc) are unaffected, so b47656b still holds. adds anon_managed_blockvar_01 regression test.
…inator append_else walks the if-chain assuming every `t1` is another ifn; once a `_:` branch deposited a non-ifn tail, a trailing else/otherwise dereferenced past it. Reject the redundancy with an error instead. In expression mode `else <expr>` or `otherwise <expr>` terminates the match itself - the explicit `end` is only required when there is no else/otherwise clause.
Previously emitted `Syntax error, "ELSE" expected but "END" found` which doesn't mention that `otherwise` is also accepted.
When the host compiler is x86_64 without `FPC_HAS_TYPE_EXTENDED` (e.g. self-built on win64), the assembler writer in `assemble.pas` rejects 80-bit float constants with IE 2015030501. The existing safeguard only covers i386 cross-compiles. Auto-define `FPC_SOFT_FPUX80` here so the soft x80 emulation path is taken whenever the host can't natively represent extended.
Mirror of i386-nativent for x86_64, registered as `system_x86_64_nativent` (#127). Sysinfo uses Win64-style settings: `cpu_x86_64`, `as_x86_64_pecoff`, 16-byte stack/parm alignment, x86_64 llvm datalayout. Linker entry names keep the underscore prefix (`_NtDriverEntry` / `_NtProcessStartup` / `_DLLMainStartup`), matching how win64 RTL declares its CRT startups. Treated like win64 for fpu types: `pbestrealtype` -> `s64floattype`, no separate `Comp` type, no `FPC_HAS_TYPE_EXTENDED`. This keeps the generated code free of legacy x87 80-bit operations that the win64 ABI also avoids. Target is flagged `tf_under_development` (upstream FPC has only the i386 variant); NDK / DDK header structs are still i386-shaped, so runtime correctness on real x86_64 NT is not guaranteed yet.
Add `x86_64-nativent` to `MAKEFILETARGETS` and duplicate the target-specific blocks (`TARGET_UNITS`, `TARGET_IMPLICITUNITS`, `TARGET_RSTS`, `COMPILER_INCLUDEDIR`, `COMPILER_SOURCEDIR`). Produces 79 .ppu (system, sysutils, classes, ndk, ddk and the rest).
Flip the `nativent` row in `OSCpuPossible` so `x86_64` is allowed alongside `i386`. Without this `fpcmake` refuses to regenerate Makefiles for `x86_64-nativent` and `make install` fails before it reaches the compile stage. `rtl/Makefile` is the regenerated artifact: `x86_64-nativent` joins `MAKEFILETARGETS` and the `TARGET_DIRS+=nativent` / `TARGET_DIRS_NATIVENT` blocks pick up the new pair.
PE32+ format requires 8-byte IAT/INT thunks (ULONGLONG per entry). The set `systems_peoptplus` gates that behaviour in `ogcoff.pas` (import generation), `assemble.pas` (aitconst_rva_symbol width) and `aasmtai.pas` (rva symbol size). x86_64-nativent was missing, so the linker emitted 4-byte thunks and `.idata` parsed by any PE viewer collapsed into duplicated names and "<corrupt>" tails - the upper half of each 8-byte slot got read as a separate 4-byte entry. After this driver imports parse cleanly: ExAllocatePoolWithTag, ExFreePoolWithTag, DbgPrint.
Driver code generated for x86_64-nativent was passing arguments via RDI/RSI/RDX/RCX/R8/R9 (SysV) while the Windows kernel calls drivers with the MS x64 convention (RCX/RDX/R8/R9 + 32-byte shadow space). The first argument to `DriverEntry` arrived in RDI, but the body read it from RCX -> kernel followed a garbage `DRIVER_OBJECT*` and the box froze on load. Introduce `systems_x86_64_ms_abi = [system_x86_64_win64,system_x86_64_nativent]` and route the ABI checks in `cgcpu.pas` (prologue/epilogue, shadow space, XMM saves) and `cpupi.pas` (`x86_64_use_ms_abi`, `set_first_temp_offset`, `generate_parameter_info`) through it. SEH64 (table-based unwind in nx64flw.pas) stays gated on `system_x86_64_win64` alone - the nativent RTL has no `_fpc_local_unwind` yet, so emitting Win64 unwind directives there would break linking.
…oaden pure eligibility to const-aggregate readers Two follow-ups to the landed -OoPURE interprocedural pure/const discovery: * -OoGVNPRE now value-numbers calls to routines proven PURE by -OoPURE. A direct, non-method call to a pure routine (reads but never writes global state, no I/O, cannot raise/trap) whose arguments are side-effect-free is treated as a memory-reading GVN candidate, keyed on its argument's safe locals. Two structurally-identical such calls with the arguments and the pure-read memory unchanged in between are commoned to a single call reused from a temp. A pure call is no longer a straight-line barrier (new gvn_has_sideeffects), so `x := f(a)` participates; its arguments are still scanned so a side effect hiding in one re-arms the barrier. Kill semantics ride the existing gvn_mem machinery: any store to memory, an impure call, or a loop that writes memory invalidates the value. Gated on -OoPURE; inert when the pure pass did not run. * -OoPURE eligibility broadened: a routine that reads through a const/constref aggregate parameter (unmanaged record / fixed array / set) but never writes it is now a PURE reader -- its field/element reads are pure memory reads and a const parameter cannot be written. Result type stays a simple scalar and methods (a mutable self alias) remain out of scope, keeping the gates conservative. These are pure-not-const, so they feed the new GVN-PRE CSE (LICM, which needs const, is unaffected). Tests: optgvnpre_purecall_01 (pure-call CSE correctness oracle: full/partial redundancy plus kill-on-arg-write and kill-on-global-store soundness), optgvnpre_purecall_disabled_01 (disabled-switch parity), optpure_constaggr_01 (const record/array reader eligibility, memory-kill soundness, var-aggregate stays impure). Full unleashed suite green (968 pass / 8 known-skip / 0 fail); unrolldyn_check.sh and slp_codegen_check.sh still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… bodies via jump thunks Ports the gcc -fipa-icf / gold --icf idea to FPC as an opt-in -OoICF pass that runs intra-unit at the assembler-list level, after every routine of a module has been generated into al_procedures and before the object file is emitted (hook in create_objectfile, pmodules.pas). For each routine it canonicalizes the body (opcode/opsize/condition, every operand encoding, memory references with base/index/scale/segment/offset/ refaddr) with the routine's own symbol and local labels symbolized to positional tokens and other relocations abstracted to the referenced symbol name. Routines with equal canonical signatures emit byte-identical machine code, so the first is kept and each later duplicate has its instruction body replaced by a single `jmp <kept>` thunk. The fold is always a thunk, never a symbol alias: every duplicate keeps its own distinct symbol and therefore its own distinct address, so @f<>@g survives even when a folded routine's address is taken and compared, and FPC's non-DWARF exception model is unaffected. All labels are preserved so the already-generated debug/CFI list never dangles. Conservative: only straight-line bodies (no embedded data/cfi/unhandled operand kinds) fold, and only when the routine has enough instructions that a 5-byte thunk is a net shrink. x86 only; a no-op elsewhere. Opt-in; NOT part of the -O4 defaults. New cs_opt_icf switch (globtype.pas) + 'ICF' name + ppudump table entry. Tests under unleashed/tests/testfiles/opticf: two identical routines fold (a runtime probe asserts exactly one begins with the 0xE9 jmp opcode) while a near-identical routine does not; addresses stay pairwise distinct; results match ICF on vs off; and a generics-instantiation case. Full suite green (972 pass, 7 pre-existing ignores) both normally and with -OoICF forced on globally; unrolldyn_check.sh and slp_codegen_check.sh still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cal dynamic arrays
Intraprocedural escape analysis that proves a local dynamic-array variable
never outlives its routine and replaces its heap buffer with a frame-local
stack buffer, eliding the fpc_getmem/fpc_dynarray_setlength call. This is the
HotSpot / gcc -fipa-pta stack-allocation idea, the safe dynamic-array subset.
A local dynamic-array var A of a NON-managed element type qualifies when its
ONLY SetLength is SetLength(A,N) with N a positive compile-time constant and
header+payload within a 4 KiB frame budget, and A provably never escapes:
it appears only as A[i], as the operand of Length/High/Low or of that one
SetLength, is never address-taken, captured by a nested routine, assigned
to/from another location, or passed whole to a callee. Recursive routines,
routines using exceptions and assembler routines are disqualified.
The rewrite runs BEFORE do_firstpass (next to -OoREFELIDE), while SetLength is
still an in_setlength_x inline node and A[i]/Length are plain nodes. It
allocates a hidden local record record refcount,high:sizeint; data:array[
0..N-1] of elem end and lowers the SetLength (guarded by if A=nil so loop
re-execution stays the RTL's SetLength-to-same-length no-op) to:
FillChar(R,sizeof(R),0); R.refcount:=-1; R.high:=N-1;
PPtrUInt(@A)^ := PtrUInt(@R.data);
refcount:=-1 is exactly FPC's own constant/read-only dynarray header sentinel
(aasmcnst begin_dynarray_const), so the scope-end finalization the compiler
still emits for A (fpc_dynarray_decr_ref) neither frees nor finalizes it, and
fpc_dynarray_incr_ref treats it as constant. The stack buffer has precisely
the heap buffer's lifetime, so behaviour is bit-identical.
Follows the established fork pattern: cs_opt_stackalloc in globtype.pas (+
OptimizerSwitchStr and ppudump table), OptimizeStackAlloc in optloop.pas,
call site in psub.pas. Opt-in (-OoSTACKALLOC); NOT part of the -O4 defaults.
Tests: unleashed/tests/stackalloc_codegen_check.sh asserts the non-escaping
array emits no allocator call (FILLCHAR instead) while a returned one still
allocates; testfiles/stackalloc/ covers non-escape zero-fill/index/Length
semantics, the loop-reexecution guard, each escape route (returned, stored to
global, aliased) staying heap-allocated and correct, managed-element arrays
not transformed, and by-var element passing. Full suite 978/985 (only the 7
known pre-existing failures) both normally and with -OoSTACKALLOC forced on
globally; unrolldyn_check.sh and slp_codegen_check.sh still pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…caller-save spills to a callee's proven volatile clobber set
Ports the idea of gcc's -fipa-ra to FPC, intra-unit and x86_64 only. When a
routine's code is generated its body is already register-allocated, so the
register allocator knows exactly which physical registers it uses
(rg[bank].used_in_proc -- the same set FPC uses to pick prologue callee-saved
pushes). optipara.RecordProcClobbers snapshots, per register bank, the
intersection of that set with the ABI volatile (caller-saved) mask and records
it on the tprocdef. That summary is already transitively complete: every inner
call allocates its callee's clobbers around itself, so a routine calling an
un-analysed target ends up with the full mask (no unsafe reduction downstream);
one that only calls small proven leaves inherits just their small sets.
Consumer side (tcgcallnode.pass_generate_code): for a DIRECT call to an
already-recorded routine, the volatile registers allocated around the call are
narrowed to the callee's clobber set, so caller values living in untouched
volatile registers survive the call without a spill/reload. On a small-leaf hot
loop this removes the rbx/r12/r13/r14 evacuation entirely.
Both integer and XMM/MM clobbers are tracked. Conservative fallbacks to the full
ABI mask (a too-small set silently corrupts caller state):
- indirect/procvar, virtual, method-pointer, external, syscall, interrupt and
assembler callees;
- routines not yet generated in this compilation (forward order);
- inline-asm-containing or exception-using callees (marked ipara_full);
- and, crucially, any call inside a caller that itself has exception handling
or an implicit finally -- an exception unwind (longjmp) restores only
callee-saved registers, so a caller value kept in a volatile register across
a call that raises and is caught here would be lost.
The per-procdef summary is transient (never written to / read from the ppu), so
on load a routine defaults to "not analysed" => callers use the full ABI mask.
Gated behind -OoIPARA; NOT added to -O4 this round. Adds the cs_opt_ipara switch
(globtype OptimizerSwitchStr + ppudump table), an ipara_codegen_check.sh
assertion and runtime regression tests under testfiles/optipara/ (int leaves,
XMM leaves, and every fallback path). Full unleashed suite: no new failures with
the switch off or forced on globally.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t-in) Port of gcc's -ftree-scev-cprop (scalar-evolution final-value replacement) plus the whole-loop deletion its control-dependent DCE performs. New optimizer unit compiler/optfinalvalue.pas, wired as opt-in -OoFINALVALUE (globtype.pas enum + name table, ppudump.pp mirror, psub.pas call site before the DFA/loop passes so the still-structured for-nodes' bounds are read directly). NOT in the -O4 defaults. Transform: a counted for-loop whose whole body is a single loop-invariant accumulator update of a plain local integer (inc(s,c) / dec / s:=s+c / s:=s-c), whose counter's exit value is dead, is replaced by the closed form of s's exit value if a<=b then s:=s+(b-a+1)*c (to<=from for downto) and the loop deleted; an empty-bodied dead counted loop is deleted outright. Post-loop uses of s then read the closed form directly. Sound conservative subset (correctness over coverage): integer inductions only; counter a plain local integer <=32 bits (so the symbolic trip count b-a+1 is exact in 64-bit); accumulator a plain local integer distinct from the counter; c and the bounds loop-invariant, side-effect-free and independent of i/s; counter/accumulator rejected if address-taken, captured by a nested scope or volatile; zero-trip loops handled by the a<=b guard; two's-complement wraparound preserved so the result is bit-identical, hence DISABLED under -Co/-Cr; break/continue/exit/call/store bodies never match. Tests: unleashed/tests/testfiles/optfinalvalue/ (ascending/downto/zero-trip accumulators, byte/word/shortint wraparound, invariant step, empty-loop deletion, break/global/live-counter/nested-capture/addr-taken no-transform cases, switch round-trip, -Co disable) all self-checking bit-exact vs a direct reference; finalvalue_check.sh proves at the assembly level that eligible loops lose their back-edge while rejected loops keep it, and that ON/OFF output is identical. Full unleashed suite: 991 pass / 6 known-ignored, both normally and with -OoFINALVALUE forced on every test; ipara/slp/stackalloc/unrolldyn checks still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add one generic per-procdef "optimizer summary" blob persisted through the PPU, then consume it for cross-unit -OoPURE (pure/const CSE of used-unit calls) and cross-unit -OoIPARA (narrowing caller-save spills around a used-unit leaf call). Mechanism (compiler/symdef.pas tprocdef.write_optimizer_summary / load_optimizer_summary): a self-describing (tag,len,payload) list streamed inside the ibprocdef entry, terminated by optsum_end. An older/absent summary yields no tags -> the existing conservative fallback. Tags live in ppu.pas (optsum_pure/optsum_ipara, optsum_icf reserved). Framing is versioned once via CurrentPPULongVersion 30 -> 31; new per-pass payloads are just new tags. - PURE: the DERIVED pure/const verdict is written as two booleans (computed at write time via a hook optpure registers, so the whole defining unit is already analysed). A used-unit routine carries pure_ppu_valid + the verdict; dfs_pure treats it as a leaf with that verdict. Kept in the interface crc so a flipped verdict recompiles dependents. - IPARA: the volatile-register clobber mask is written guarded by a target/ABI/-Cf instruction-set signature; a mismatching or full summary is dropped to the full ABI mask. The ncgcal consumer already had no same-unit gate, so cross-unit narrowing works once the mask is loaded. - ppudump dumps the new blob (PURE verdict, IPARA signature + mask, unknown tags skipped by length). Tests: unleashed/tests/optsummary/ two-unit fixtures + optsummary_check.sh (cross-unit pure call CSEs 3->1 under -OoGVNPRE -OoPURE, absent summary keeps 3, bit-exact/correct in every config); ipara_codegen_check.sh extended with a cross-unit case (used-unit leaf narrows callee-saved pushes 4->0, falls back when the callee unit lacks the switch). Full unleashed suite green normally and with -OoPURE -OoIPARA -OoGVNPRE forced on (992 pass / 0 unexpected / 5 ignored); RTL rebuilt for the version bump; all *_check.sh pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two follow-ups to the intra-unit -OoICF pass (compiler/opticf.pas):
1. Symbol-alias mode. A proven address-never-observed duplicate is now
collapsed to a *zero-byte* symbol alias (its entry symbol emitted as an
additional label at the survivor's address, the same two-labels-one-location
mechanism the codegen already uses for main/PASCALMAIN) instead of a 5-byte
jmp thunk. Gated conservatively via a new source-level address-taken proof:
- tprocdef.icf_addrtaken, set at the canonical proc->procvar conversion
(ttypeconvnode.create_proc_to_procvar) and at tloadnode.create_procvar,
captures every @proc / procvar assignment / procvar argument;
- the routine must be source-invisible to other units (procsym in the
static/local symtable, not exported/public/external/virtual, not a
method), so no source anywhere can form @dup;
- single entry symbol only (multi-aliasname routines fall back to thunk).
Address-taken / interface-visible / method duplicates keep the
address-preserving thunk, so @f<>@g stays observable where guaranteed. We do
NOT gate on the ELF global bind: under default section-per-symbol
smartlinking every routine symbol is emitted global, yet an
implementation-section routine is still un-nameable from another unit.
2. Cross-unit folding via the shared PPU optimizer-summary mechanism. Each
globally-visible routine that survives intra-unit folding stores a 128-bit
digest of its full canonical instruction stream on its tprocdef, persisted as
the (previously reserved) optsum_icf ppu tag guarded by the IPARA-style
target/ABI/-Cf signature and carrying the survivor's global name (captured at
write time; mangledname is not yet resolvable at load). A later unit registers
every used unit's survivors and folds a digest-matching routine into a jmp
thunk to that external global symbol. Cross-unit folds are always thunks
(never aliases): the survivor lives in another object and a thunk keeps every
duplicate's own address intact. Collision stance: cross-unit we trust the
128-bit digest of the full canon alone (no body to compare) -- birthday-bound
~n^2/2^128, and the identical canonicalisation + signature guard must also
match; documented in the unit header. The intra-unit bucket now keys on the
digest and still byte-verifies via full canon compare (fixing latent
truncation of >255-char canons, since mangled names are shortstrings here).
No PPU version bump (new length-prefixed, skippable summary tag). Survivors are
kept alive under smartlinking by the thunk/alias reference; verified with
default flags and -XX -CX -Xs.
Tests: unleashed/tests/testfiles/opticf/opticf_alias_01.pp (alias correctness);
unleashed/tests/icf_check.sh + icf/ fixtures (zero-byte alias asm shape,
address-taken thunk fallback with @ping<>@pong, cross-unit external jmp,
non-matching bodies never fold, bit-exact vs ICF-off and under smartlink).
Full unleashed suite: 992 pass / 0 unexpected. With -OoICF forced on: only
opticf_disabled_01 "fails" (it asserts ICF-off, so forcing it on is by design).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend the -OoVECTORIZE loop autovectorizer from single-only to also handle `double` dynamic arrays, 2 lanes per 128-bit window. Every existing shape now fires on double: array-array a[i]:=b[i] op c[i] (+ - *), scalar-broadcast a[i]:=b[i] op s / s op b[i] (incl. non-commutative subtraction and constant literal), plain copy a[i]:=b[i], and the sum / dot-product reductions (including the FMA-contracted form) under fast-math. Backend: tvectoropnode gains an `isdouble` flag (nbas.pas); the x86 emitter (nx86inl.pas) selects movupd/addpd/subpd/mulpd, movsd+unpcklpd for the splat, xorpd+movsd for the reduction seed, and an unpckhpd+addsd 2-lane horizontal sum, with the VEX v-forms under an AVX fputype (still 128-bit). min/max if-conversion stays single-precision only. Recognizer (optloop.pas): the precision is fixed from the destination array (store shapes) or the accumulator (reductions); every source array, scalar operand and RHS/addend must match it, so any single/double mix is rejected and stays scalar (a wrong lane width would miscompile). All single-precision soundness gates are preserved unchanged: plain-counter index, -Cr/-Co disable, globals/address-taken/volatile scalar rejection, self-alias safety, bit-exact vs the scalar path (incl. NaN/Inf/-0.0 and tail residues), and the fast-math gate on reductions. The SLP straight-line vectorizer stays single-only. AVX-256 (ymm) widening is intentionally NOT taken here: it needs a 256-bit horizontal-reduction path and OS_M256 codegen across the node, which is invasive/higher-risk; left as a documented follow-up. Tests: unleashed/tests/testfiles/optvect/vect_double_* (10 runtime bit-exact tests mirroring the single ones: per-op array-array, scalar-broadcast, copy, sum+dot reductions, store & reduction special-values, self-alias, mixed-type rejection, -Cr disable, AVX) plus vectorize_double_check.sh (asserts double loops pack to addpd/movupd, stay scalar with the switch off, and that a mixed single/double loop never vectorizes). Full unleashed suite green normally and with -OoVECTORIZE forced on (1000 pass / 0 unexpected). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(d) Emit a -vh hint per analyzed routine when -OoPURE discovers it pure/const (messages 06087/06088, once at AnalyzeProcPurity time, gated by hint verbosity). Reports the strongest verdict (const implies pure). (e) Broaden purity eligibility to read-only instance/class methods (the classic function TFoo.GetX:Integer;begin Result:=FX end accessor): - proc_eligible now accepts non-virtual, non-abstract, non-ctor/dtor methods; self is skipped as a read-only reference parameter. - lvalue_write_is_side_effect treats ANY store reachable through self as a side effect regardless of how self is passed (a by-value class pointer or a by-ref record self), so a method that writes a field is correctly impure -- the soundness-critical gate. - A method reads self's referenced memory, so it is at best "pure", never "const" (pure_reads_global stays set). No new PPU summary flavour is needed: the "pure" bit already means memory-dependent, and the -OoGVNPRE consumer treats every pure call as a memory reader. -OoGVNPRE method-call CSE (memory dependence): gvn_pure_call now accepts a provably-direct pure method call (non-virtual/abstract procdef, side-effect-free methodpointer); gvn_candidate value-numbers it as gvn_mem keyed on the locals read by self AND the arguments. Any intervening store (including a field write to the object) or any call therefore invalidates it -- two getter calls with a field write or an impure call between them are NOT commoned; two with nothing between are. Virtual/abstract methods are never folded. Cross-unit works via the existing optsum_pure bit (a pure method serializes exactly like a pure function). Tests: optpure_method_01.pp (bit-exact runtime: record-self getter commons, class getter with intervening field write / call / object reassignment does not); pure_method_check.sh (codegen call-count assertions incl. virtual-not-folded); pure_hint_check.sh (verdict text, silent for impure and with -OoPURE off). Full unleashed suite 1003 pass / 0 unexpected, bit-clean normally and with -OoPURE -OoGVNPRE forced on; all *_check.sh green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The level2 range-test optimization in nadd.pas (is_range_test) rewrites (C1 <= expr) and (expr < C2) into the unsigned test (expr - C1) <= (C2 - C1). When the lower bound C1 is negative, the low-bound constant is stored as an nf_internal *unsigned* ordinal (adaptrange converts e.g. -149 to qword(-149) = $FFFFFFFFFFFFFF6B). When such a helper is inlined at a call site where "expr" folds to a compile-time constant, the const-fold in taddnode.simplify evaluated e.g. 0 - qword(-149) which sets the tconstexprint overflow flag (0 < $FF..F6B in unsigned arithmetic) and raised a spurious user-facing "Overflow in arithmetic operation" ERROR at any level enabling cs_opt_level2 (-O2/-O3/-O4) -- even though this is intentional modulo-2^n wraparound in compiler-generated code. This made pletora/neural-api's neuralvolume.pas uncompilable at -O2+: inside pcr_powf -> is_exact_pf the range test "(-149 <= Trunc(y)*e) and (...)" folded once y=1/2.4 gave Trunc(y)=0, and the error surfaced (via the inlined body's stored fileinfo) at lab2rgb's three float "/const" statements, neuralvolume.pas(2147,1). Fix: in the addn/subn/muln integer const-fold, do not raise parser_e_arithmetic_operation_overflow for nf_internal nodes; keep the wrapped value (create_simplified_ord_const masks it to the result type). User-level overflow reporting is unchanged for non-internal nodes. Regression test const_fold_range_test_neg_lowbound_01 fails to compile at -O2 with the fix reverted (clean at -O1) and compiles+runs correctly with the fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Port gcc's -foptimize-sibling-calls as an opt-in -OoSIBCALL pass, distinct from
the self-recursion loop rewrite (-OoTAILREC/cs_opt_tailrecursion) and from
partial inlining. When a routine's last action is a direct tail call to ANOTHER
routine (result := OtherFunc(args) or a bare tail procedure call), the frame
teardown is hoisted above the call and the call is rewritten to a jmp, so the
callee reuses the caller's return slot: O(depth) stack becomes O(1) for mutually
recursive pairs and continuation-style dispatch.
Implemented at the x86-64 assembler-peephole level (TX86AsmOptimizer.OptSibCall,
run from PostPeepholeOptCall, gated on cs_opt_sibcall). It also handles the
result-forwarded-through-a-callee-saved-register shape that the -O4
CallFrameRet2Jmp deliberately punts on -- the shape the register allocator emits
for `result := OtherFunc(args)`, i.e. exactly the mutual-recursion case:
recognises call X / mov %rax,%cs / [shared label] / mov %cs,%rax / <rsp
release + callee-saved pops> / ret, hoists only the pops+release (never the
result movs -- the jmp leaves the result in rax), and jmps to X.
Sound subset (x86-64 SysV/Linux only; anything not provably safe falls back to a
plain call):
* direct call to a symbol (indirect/procvar rejected);
* teardown = leaq/addq rsp release and/or pops of callee-saved integer regs;
* optional single result forward rax->callee-saved->rax (identity);
* no outgoing stack arguments in the routine (maxpushedparasize=0), so a
callee whose stack-arg area exceeds our zero incoming area is excluded;
* caller convention register/cdecl/stdcall (safecall + exotic excluded);
* CurrentProcAllowsSiblingTailFrameReuse: no try/finally/exception frame, no
dynamic stack alloc, no nested-frame capture, no address-taken local/param
(so no @Local can escape into the callee's arguments).
Opt-in; NOT added to genericlevel4optimizerswitches.
Switch wired in globtype.pas (cs_opt_sibcall + 'SIBCALL') and the ppudump name
table; -Oo parsing is table-driven.
Tests: testfiles/sibcall/ (even/odd + continuation runtime correctness, and a
disqualifier correctness file) pass under the suite both normally and with
-OoSIBCALL forced on the whole suite (1006 pass / 6 known pre-existing fails,
unchanged either way). unleashed/tests/sibcall_check.sh asserts the jmp in the
eligible case, a plain call with the switch off, and a plain call for each of
the four disqualifiers. An even/odd pair to depth 10^7 completes under
ulimit -s 512 with the flag and segfaults without it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extends the opt-in final-value-replacement / dead-loop-elimination pass with two sound follow-ups: * POINTER-STRIDE accumulators: a counted loop whose body advances a plain local pointer by a loop-invariant stride via inc(p,stride)/dec(p,stride)/ inc(p) is replaced by the closed form p := p + (b-a+1)*stride, built as a pointer+integer add so the compiler applies the same element-size scaling the loop body did. The still-unlowered inc/dec inline form carries the raw (unscaled) element stride; the p:=p+stride assignment form is deliberately rejected because by this point the stride is already element-size-scaled and rebuilding a pointer+integer closed form would scale it a second time. * MULTI-STATEMENT bodies: a body of several statements, each an independent accumulator update of a DISTINCT plain local (integer or pointer), matches -- one closed form per accumulator under the shared trip-count guard. Two updates of the same accumulator (double-count) reject the loop. Every increment must be loop-invariant and reference neither the counter nor any accumulator, so the accumulators evolve independently and statement order is irrelevant. All prior soundness gates (<=32-bit counter, dead counter, no overflow/range checking, zero-trip guard, two's-complement wraparound) are preserved. Still opt-in; not in the -O4 defaults. Tests: finalvalue_check.sh grows codegen assertions (pointer-inc and multi- accumulator loops deleted; duplicate-accumulator and pointer-assign loops kept). New runtime fixtures fv_pointer_stride_01 (typed + PChar pointers, unit and non-unit strides, inc/dec, ascending/downto, zero-trip) and fv_multi_accum_01 (two/three integer accumulators, mixed integer+pointer body), plus dup-accumulator and pointer-assign kept-loop cases in fv_no_transform_01, all checked bit-exact against a direct reference and ON/OFF byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e elimination The extended (record-field / static-array-element) DSE in optdeadstore.pas treated every call as a hard barrier that flushed the whole pending-store set. With -OoPURE on, a call whose resolved direct target carries a pure or const verdict is now relaxed, precise about the pure-vs-const asymmetry: * const call: reads and writes no memory -> full non-barrier; pending stores survive (only its argument expressions, scanned normally, perform reads); * pure call: writes nothing but MAY READ global/heap state -> treated as a potential reader: pending stores to globally reachable slots (static vars, by-ref const params) are kept live, while stores to non-address-taken locals and by-value params (which no callee can name) still survive; * impure / indirect / procvar / virtual / aggregate-returning calls remain full barriers, as do deref and asm nodes. Soundness first. Both the non-candidate-assignment scan and the previously unconditional "any other statement" barrier for bare call statements go through the new relaxed path (el_classify_call / el_haz_scan / apply_relaxed_barrier). Tests: unleashed/tests/pure_dse_check.sh asserts on codegen that the dead store IS removed across a const call and across a pure call on a local base, and is NOT removed across an impure call or across a pure call on a static base, and that the baseline (no -OoPURE) keeps everything; runtime fixture testfiles/optpure/optpure_dse_01.pp is deterministic and self-checking at -O1/-O2/-O3+flags/-O4+flags. Full suite 1009 pass / 6 known pre-existing fails normally AND with -OoPURE forced suite-wide; all 12 *_check.sh PASS. (The 7 optvect failures under forced -Oodeadstore are a pre-existing -Oodeadstore+VECTORIZE-reduction interaction, reproduced with this change stashed; not introduced here.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tale accumulator seed)
Root cause
----------
The autovectorizer's reduction path (OptimizeVectorize, vok_reduce_sum /
vok_reduce_dot in optloop.pas) seeds lane 0 of the packed accumulator with the
incoming scalar accumulator s -- i.e. the user's s := <start> store just
before the loop. That seed read of s was embedded directly inside the
create_reduce_init backend (tvectoropnode) node:
acc := reduce_init( acctemp, load(s) )
A backend node's operand reads are not reliably modelled by data-flow analysis
(the finish/horizontal-sum store already worked around this by writing s with a
plain assignment from a temp, per the comment at the store site). Because DFA
did not see s being read at the seed, the incoming s := <start> def looked
dead, and with dead-store elimination enabled (-Oodeadstore) the stock DSE pass
in optdeadstore.pas removed it. Lane 0 was then seeded from a stale stack slot,
producing a wrong sum / dot-product result.
This is why forcing -Oodeadstore suite-wide made 7 optvect runtime tests fail
(vect_reduce_01, vect_reduce_avx_01, vect_reduce_disabled_01,
vect_reduce_special_values_01, vect_double_avx_01, vect_double_reduce_01,
vect_double_reduce_special_values_01) while the compiler was otherwise correct
without -Oodeadstore. The bug is pre-existing and independent of -OoPURE.
Fix
---
Mirror the store-side workaround for the read side: copy the incoming s into a
memory-backed scalar temp with a PLAIN assignment first, and seed the packed
accumulator from that temp:
seedtemp := s; { DFA-visible read of s keeps its def live }
acc := reduce_init( acctemp, seedtemp )
The plain seedtemp := s assignment is an ordinary loadn read of s that DFA
tracks, so the incoming def of s stays live and DSE no longer removes it. Emitted
code is otherwise identical (one extra scalar copy that the register allocator
folds away in the common case).
Test
----
New runtime fixture unleashed/tests/testfiles/optvect/vect_reduce_deadstore_01.pp
forces -OoVECTORIZE -Oodeadstore via %OPT and checks single/double sum and dot
reductions against a strict sequential (downto) oracle over exactly-representable
inputs for every trip count; it fails before this change and passes after.
Full unleashed suite bit-clean both normally and with -Oodeadstore -OoVECTORIZE
forced suite-wide (1011 pass / 5 pre-existing known fails, no new failures); all
30 optvect tests green with the flags forced; all 12 *_check.sh codegen scripts
PASS.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…atic/global accumulators Lands the three remaining -OoFINALVALUE follow-ups, soundness-first (a wrong final value or wrongly-deleted loop is a miscompile, so the pass still leaves any non-provable loop alone). * 64-BIT COUNTERS: the counter may now be any integer up to 64 bits (max size raised 4 -> 8). The exact trip count is not needed: b-a+1 is computed in 64-bit two's-complement (the true iteration count reduced mod 2^64) and each accumulator product is truncated to the accumulator's own width w<=64, so (count*c) mod 2^w is bit-identical to the repeated addition; a pointer stride's count*stride element count is likewise correct mod 2^64. * STATIC / GLOBAL accumulators: a unit-level (static) variable may be an accumulator (never the counter -- a static counter's cross-unit uses the whole-routine deadness scan cannot see). Sound because the matched body has no calls and the routine no exception paths, so nothing observes the global's intermediate states cross-routine within this thread; the closed form stores the identical final value. different_scope (normally set on any global reached from a subroutine) is no longer treated as the nested-capture hazard -- that check now applies to locals only; addr_taken / volatile still reject, and threadvars / externals stay out. * RANGE CHECKING (-Cr, without -Co): re-enabled, restricted to native full-range integer accumulators (int64/qword) whose s:=s+c performs no narrowing and so is never range-checked, matching the nf_internal closed form bit-for-bit. Sub-native / subrange / pointer accumulators are range-checked on their narrowing store, so they are declined (loop kept). Bound range faults are preserved (the guard copies the original bound expressions verbatim), and the empty-dead-loop shortcut is disabled under -Cr so a narrowing bound still faults at loop entry. * -Co (overflow checking) stays DISABLED by design: a native accumulator loop traps on the overflowing iteration while the wrap-around closed form does not, and reproducing that per-iteration trap from a single closed-form step is not robustly sound (sub-native trap-freedom relies on a fragile promotion detail; a native accumulator's count*c can overflow int64 in exactly the trapping case). The pass exits early whenever overflow checking is on. Tests: new runtime fixtures fv_counter_64bit_01, fv_global_accum_01, fv_rangecheck_native_01; finalvalue_check.sh extended with codegen asserts (64-bit-counter / native / global loops deleted, deleted under -Cr; sub-native kept under -Cr; every loop kept under -Co). fv_no_transform_01 updated (global accumulator is now transformed). Full unleashed suite 1013 pass / 0 fail, byte-identical normally and with -OoFINALVALUE forced suite-wide. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend TX86AsmOptimizer.OptSibCall to recognise result forwarding through more than one location. The old matcher accepted only a single callee-saved register identity pair (rax->cs->rax). It now walks a chain of forward-saves that stage one or more integer return registers (rax, or rax:rdx for a 128-bit result) through callee-saved registers and/or rsp-relative frame slots, each with a matching restore before the teardown. Sources and destinations must be pairwise distinct and every save must be restored before any teardown, so the save/restore round-trip is a proven identity on the return registers; the transform then hoists the teardown, turns the call into a jmp, and drops the (now dead) forwarding moves exactly as before. This is the shape the register allocator emits for a 128-bit rax:rdx record result staged through two frame slots, and for a result split across a callee-saved reg and a stack slot. Also harden a pre-existing soundness hole in the frame-teardown gate (CurrentProcAllowsSiblingTailFrameReuse): symtable_has_addrtaken only tracks named locals/parameters, so compiler temporaries whose address escapes to the callee -- an open array of const, a @temp passed by reference -- slipped through and the frame was torn down before the callee read them (dangling stack). Scan the routine and decline when any lea/mov materialises a frame-relative address into a general register. This fixes a real miscompile that surfaced only under a forced suite-wide -O2 -OoSIBCALL (match_as_expression_in_arg_01); it does not affect the eligible register-only mutual-recursion / continuation cases. Tests: sibcall_forward128_01.pp (deep 128-bit + single-reg forwarding proving O(1) stack and correct results), sibcall_frameaddr_negative_01.pp (escape cases decline and stay correct), and new firing/decline codegen assertions in sibcall_check.sh (multi-location forward jmps, non-tail and frame-address declines). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…icter case-exhaustiveness check in optloop.pas Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- ProducedExePath appended .exe unconditionally, but unix binaries have no extension: every run-phase test reported 'compiled but exe not found' - inject -Facthreads on non-Windows targets so the threaded-feature tests (parallel for, async/await, lock, thread static) get the cthreads driver the compiler itself only hints about With these, 1200/1202 tests pass on x86_64-linux; the two failures (inline_vars widestring RTTI name, asyncawait reraise flake) reproduce identically on the pre-merge compiler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fibodevy
left a comment
There was a problem hiding this comment.
"Adds regression tests under unleashed/tests/testfiles/dead_store_normalize/ (all fail on the old compiler)"
All four pass on current main (-O3 -OoDEADSTORE):
[1/4] dead_store_normalize\dsnorm_ifexpr_nested_loops_01.pp ... PASS
[2/4] dead_store_normalize\dsnorm_match_in_loop_01.pp ... PASS
[3/4] dead_store_normalize\dsnorm_ifexpr_in_loop_01.pp ... PASS
[4/4] dead_store_normalize\dsnorm_funcref_compiles_01.pp ... PASS
The IE 200108231 fix landed upstream about a month ago (!1442), so the normalize() bug has been gone there since then. Today's sync pulled it into our main (acc89a1), so as of today it's fixed on our side too - which is why these pass instead of fail. "all fail on the old compiler" only held against the pre-sync base.
So the fix part is redundant now - unless I've overlooked something, in which case can you point to the case it still handles that acc89a1 doesn't? The regression tests themselves are welcome (they pass and lock the fix in), so I'd just take those.
Also worth checking the other fixes in the PR the same way - since this one was already resolved upstream, some of the others may have landed there too and be redundant after today's sync.
|
Many thanks for the feedback. I will work on it (eventually). I will review: the nadd.pas const-fold spurious overflow, the -Oodeadstore × -OoVECTORIZE stale-seed miscompile, the testtool unix run-phase fix, and the case-exhaustiveness fixup. |
Add overloaded declarations bound to the same external symbols so callers can pass a variable directly instead of a pointer for the out-parameters of clGetPlatformInfo, clEnqueueMapBuffer, clEnqueueMapImage, clGetDeviceInfo, clCreateContext, clCreateContextFromType, clGetContextInfo, clGetCommandQueueInfo, clSetCommandQueueProperty, clCreateBuffer, clCreateImage2D, clCreateImage3D, clGetSupportedImageFormats, clGetMemObjectInfo, clGetImageInfo, clCreateSampler, clGetSamplerInfo, clCreateProgramWithSource, clCreateProgramWithBinary, clGetProgramInfo, clGetProgramBuildInfo, clCreateKernel, clCreateKernelsInProgram, clGetKernelInfo, clGetEventInfo and clGetEventProfilingInfo. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a clCreateCommandQueue overload taking 'var errcode_ret: cl_int' alongside the existing pcl_int version; mark both with 'overload'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arus checkout rebuildu.sh chains the full toolchain rebuild (compiler clean cycle seeded via fpc -PB, RTL, packages, and the LCL when ../lazarus exists), detects the host CPU instead of assuming x86_64, and warns about stray package PPUs shadowing rtl/units. lazbuildu.sh now passes --lazarusdir pointing at the sibling ../lazarus checkout (LAZDIR overridable) so dependency packages are rebuilt from source with the in-tree compiler instead of failing with "PPU Invalid Version" against the system Lazarus install's precompiled units. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fpmake's NeedsCompile never compares package PPUs against the compiler or RTL (it only recompiles when a dependency package was built in the same fpmake session, or the unit's own sources/includes are newer), so a default rebuild left packages compiled by the previous compiler against the previous RTL next to the fresh toolchain. Make the packages clean unconditional; --no-clean-packages restores the old fast path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
5178d48 to
9e03840
Compare
New -O4 tree-level optimization passes, x86 peepholes, and supporting fixes
I have not properly tested. This is for you to know what I am working on.
Summary
This PR adds a family of new optimization passes to the Free Pascal compiler, enabled at
-O4(each individually controllable via its own-Oo…/-OoNO…switch) or opt-in via their own switch, together with x86-specific code-generation improvements, interprocedural analyses with cross-unit PPU summaries, bug fixes uncovered by self-compiling the compiler at-O4, and an extensive regression-test suite underunleashed/tests/(~320 files changed, ~40,000 insertions across 69 commits).New tree-level optimizer passes (
-O4)Loop optimizations (
compiler/optloop.pas, ~6,000 new lines)-OoLICM— loop-invariant code motion: hoists invariant computations out of loops.-OoLOOPUNSWITCH— loop unswitching: moves loop-invariant conditionals out of the loop, duplicating the loop body per branch (with a size cap).-OoLOOPPEEL— full peeling of small constant-trip-count loops.-OoLOOPSPLIT— loop splitting on an induction-variable crossover point.-OoLOOPFUSE— fusion of two adjacent loops iterating over the same space.-OoLOOPDISTPAT— loop-distribution pattern idiom recognition.-OoUNROLLJAM— outer-loop unroll-and-jam.-OoPREDCOM— predictive commoning of cross-iteration reloads.-OoIFCONVERT— loop if-conversion to packedmaxps/minps.-OoREASSOC— floating-point reduction reassociation (gated on fast-math).-OoUNROLLDYN(opt-in) — dynamic-trip loop unrolling: unrolls a counted for-loop of unknown trip count by 4 with a scalar remainder loop, keeping the original serial evaluation order so FP accumulation stays bit-identical.-OoPREFETCH(opt-in) — software prefetch: inserts aPREFETCHNTAofbase[i+64]once per iteration group for each distinct streamed dynamic-array base (skipping pure store destinations); semantically a no-op.-OoFINALVALUE(opt-in) — final value replacement + dead loop elimination (gcc-ftree-scev-cprop, new unitcompiler/optfinalvalue.pas): a counted for-loop whose body is only loop-invariant accumulator updates of plain locals, with a dead counter, is replaced by the closed formif a<=b then s := s + (b-a+1)*cand the loop deleted. Extended to pointer-stride accumulators (inc(p,stride)), multi-statement bodies (one closed form per distinct accumulator), 64-bit counters (exact mod 2^64 arithmetic), static/global accumulators, and native full-range accumulators under-Cr; disabled under-Coby design (the closed form cannot reproduce the per-iteration overflow trap).Vectorization
-OoVECTORIZE— conservative SSE loop autovectorization of element-wise array loops (add/mul/sub, copy, and scalar-broadcast shapes), with alias, range-check, and special-value safety guards. Includes per-loop diagnostics (notes 06065/06066) explaining why a loop was or was not vectorized.-OoVECTORIZEto the two canonical single-precision reductions,s := s + a[i](sum) ands := s + a[i]*b[i](dot product), widened to a 4-lane packed SSE/AVX accumulator with a scalar residue loop; recognizes the FMA-contracted dot shape on FMA targets. Fast-math gated (it reassociates FP adds, like-OoREASSOC). New note 06086.-OoVECTORIZEfrom single-only todoubledynamic arrays, 2 lanes per 128-bit SSE/AVX window: every existing shape (array-array+ - *, scalar-broadcast, copy, and the sum/dot reductions including the FMA-contracted form) now fires on double, withmovupd/addpd/subpd/mulpdand a 2-lane horizontal sum in the backend. Any single/double mix is rejected and stays scalar; all single-precision soundness gates are preserved.-OoSLP(opt-in) — superword-level parallelism (gcc tree-slp-vectorize / LLVM SLPVectorizer): packs a run of ≥4 adjacent, isomorphic scalar single-precision assignments over consecutive elements of the same array base — hand-unrolled straight-line code with no surrounding loop — into one 128-bit SSE packed op, reusing the loop vectorizer's backend node. Bit-identical to scalar (no reassociation, no fast-math gate).Value-range and idiom recognition
-OoRANGEELIM— value-range-analysis-driven range-check elimination (including dynamic-arrayHigh/Length-1bounds and must-check preservation for off-by-one and variable bounds).-OoVRP— interval value-range propagation with branch folding: forward-propagates integer intervals seeded from declared subrange/ordinal type bounds, for-loop counter bounds, and straight-lineconst/mod/andfacts, then folds everyifthe intervals already decide, deleting the dead arm.-OoBITIDIOM— recognition of bit-manipulation idioms: population count, andtzcnt/bsrbit-scan loops.modby a non-power-of-two constant: magic-number strength reduction (compiler/x86/nx86mat.pas).loopvar * invariantmultiplies inside array-index / pointer-offset expressions.Scalars, stores, and control flow
-OoSRA— scalar replacement of non-escaping local aggregates (new unitcompiler/optsra.pas).-OoSTOREMERGE— coalescing of adjacent narrow stores into wider stores.-OoJUMPTHREAD— jump threading / elimination of nested re-tests of the same condition.compiler/optdeadstore.pas) — DSE now also handles record-field and static-array-element stores.-OoSINK— partially-dead code sinking (the symmetric counterpart of LICM): a pure assignment immediately preceding anifwhose value is consumed on only one arm (and dead after the branch) is relocated into that arm, so paths that never use it stop paying for it.-OoSTOREMOTION— loop store motion / scalar promotion (gcc-fgcse-sm): a global of unmanaged scalar type repeatedly loaded/stored in a loop is promoted to a register temp — loaded once before the loop, stored once after — under a strict no-call/no-alias/no-trap whitelist over the entire loop.-OoGVNPRE(opt-in) — dominator-based global value numbering / full-redundancy elimination across statements and rejoining branches: a side-effect-free scalar expression available on every path to a point is computed once into a temp and reused. Availability tables are copied into branch arms and intersected at rejoins; killed conservatively by operand writes, memory stores, and calls. Validated by a 20k-case randomized differential and a self-compile with the pass active (816 firings). New note 06085.-OoPARTIALINLINE(opt-in) — partial inlining / function splitting (gcc-fpartial-inlining, new unitcompiler/optpartialinline.pas): a routine that begins with a cheap, side-effect-free early-exit guard followed by an expensive body is split into a tiny inlinable header (guard + forwarding call) that takes over the original procsym, and an out-of-line body keeping the full original code — so the common guarded-exit path pays no call at all. Covers both procedures and functions (the function case gives the header its own funcret local and remaps the guard'sexit(value)onto it, so both arms write the same result location). A 97%-early-exit guard in a hot loop runs ~16% faster.-OoSTACKALLOC(opt-in) — escape-analysis stack allocation of non-escaping local dynamic arrays: a local dynamic array of unmanaged element type whose onlySetLengthhas a positive constant length (within a 4 KiB frame budget) and which provably never escapes is given a frame-local buffer with FPC's own read-only refcount sentinel (-1), eliding the heap allocation entirely; scope-end finalization neither frees nor finalizes it.Interprocedural analyses and cross-unit PPU summaries
-OoPURE(opt-in) — interprocedural pure/const discovery (gcc-fipa-pure-const, new unitcompiler/optpure.pas): walks the unit's routines bottom-up over the call graph and proves per routine whether it is const (result depends only on by-value parameters) or pure (may read but never writes global memory, no I/O or other side effects), with a greatest-fixpoint DFS so mutually-recursive SCCs are still proven. Anything not provably harmless is impure. Eligibility covers standalone routines, const/constref-aggregate readers, and read-only instance/class methods (the classic field getter). Emits-vhpurity hints (messages 06087/06088). Consumed by:-OoLICM— hoists proven-const calls with loop-invariant arguments out of loops;-OoGVNPRE— CSEs identical pure calls (including direct pure method calls), value-numbered as memory readers killed by any intervening store or impure call;-OoIPARA(opt-in) — interprocedural register allocation (gcc-fipa-ra): records each generated routine's proven volatile-register clobber set, and narrows the caller-save spills around a direct call to that set, so caller values in untouched volatile registers survive without a spill/reload. Conservative full-ABI-mask fallbacks for indirect/virtual/external/assembler callees, exception contexts, and not-yet-generated routines.-OoICF(opt-in) — identical code folding (gcc-fipa-icf/ gold--icf): canonicalizes each generated routine body at the assembler-list level and replaces byte-identical duplicates with a 5-bytejmpthunk to the kept copy (addresses stay pairwise distinct, so@f<>@gsurvives). A proven address-never-observed, unit-private duplicate is instead collapsed to a zero-byte symbol alias at the survivor's address.(tag,len,payload)summary blob streamed inside the PPUibprocdefentry (versioned once viaCurrentPPULongVersion30 → 31; new passes add tags without further bumps), dumped byppudump. Consumed for cross-unit-OoPURE(pure/const CSE of used-unit calls, verdict kept in the interface CRC), cross-unit-OoIPARA(clobber mask guarded by a target/ABI/-Cfsignature), and cross-unit-OoICF(128-bit digests of surviving routine bodies; a digest-matching routine in a later unit folds to a jmp thunk to the external symbol).Case-statement lowering
-OoCASECLUSTER— gcc-style case clustering: partitions sorted case labels by dynamic programming into an optimal mix of jump-table clusters, bit-test clusters (thecase c of 'a','e','i','o','u'shape becomes one shift+AND membership test), and plain-compare singletons, dispatched via a balanced binary comparison tree.-OoSWITCHTABLE— case-to-lookup-table conversion (gcc-ftree-switch-conversion): a fully-covered, hole-free case whose arms only assign compile-time constants to the same ordered set of ordinal variables is replaced by per-variable static lookup tables plus a single range guard (no guard at all under full type coverage). New notes 06083/06084.x86 code generation
compiler/x86/aoptx86.pas): tail calls from a framed routine now reuse the caller's frame, with negative guards for managed types, pointer arguments, andtry/finallycontexts.-OoSIBCALL(opt-in) — sibling-call optimization (gcc-foptimize-sibling-calls), distinct from the self-recursion rewrite: when a routine's last action is a direct tail call to another routine, the frame teardown is hoisted above the call and the call rewritten to ajmp, so O(depth) stack becomes O(1) for mutual recursion and continuation-style dispatch (an even/odd pair to depth 10⁷ completes underulimit -s 512). Handles result forwarding through a chain of callee-saved registers and/or frame slots (including 128-bitrax:rdxrecord results). Hardened frame-teardown gate: anylea/movmaterialising a frame-relative address into a general register (escaping compiler temporaries such as open-array-of-const) declines the transform.-OoCROSSJUMP— cross-jumping / tail merging (gcc-fcrossjumping) as a late post-peephole pass: predecessor blocks ending in an operand-exact identical instruction tail keep one copy and jump into it. Pure code-size/I-cache win; refuses to delete regions containing labels, CFI directives, or debug/unwind data.-OoBLOCKORDER— static hot/cold basic-block layout (gcc-freorder-blocks): sinks cold error/raise regions (guard clauses ending infpc_raiseexception,RunError, range/overflow helpers, etc.) out of the hot path to the end of the routine, inverting the guard so the hot successor is the fall-through.-OoREE— redundant sign/zero-extension elimination (gcc-free) as a post-peephole pass: deletes amovzx/movsxwhose value already carries the required extension from its nearest definition (an earlier extension, anand-mask,xor reg,reg, or a small non-negative immediate move), reaching past the adjacency limit of the existing peepholes.-OoSHRINKWRAP(opt-in) — shrink-wrapping (gcc-fshrink-wrap): sinks a push-only callee-saved prologue below an initial volatile-only guard clause, so early-exit fast paths (nil check, zero-length, cache hit) run prologue-free with a bareret. DWARF CFI stays correct (position labels relocate with the pushes); validated withobjdump --dwarf=framesand a full self-host with the pass forced on.Managed types
-OoREFELIDE(opt-in) — managed refcount elision (ARC-style pair elimination): an ansistring borrowa := bwhereais thereafter only read is lowered to a plain pointer copy with the incref and the finalization decref both removed; the source provably keeps the buffer alive fora's lifetime. Verified leak-free with-ghon every fixture.Bug fixes and infrastructure
opttree: fixednormalize()hoisting block-expressions out of control-flow bodies, which broke-Oodeadstoreonif-expressions andmatchinside loops.REASSOC: fixed a miscompile / internalerror 200306031 when the loop counter appears in the accumulated expression — substituted(i+delta)copies kept stale cachedresultdef/pass1 flags; the whole substituted expression is now re-firstpassed.nadd.pasconst-fold: fixed a spurious "Overflow in arithmetic operation" error at-O2+ when the level-2 range-test rewrite'snf_internalunsigned wraparound constants (a negative low bound stored asqword) fold at an inlined call site — internal nodes now keep the wrapped value instead of raising; user-level overflow reporting is unchanged. (Madeneural-api'sneuralvolume.pascompilable at-O2+.)optvectorize: fixed a-Oodeadstore×-OoVECTORIZEreduction miscompile — the packed accumulator's lane-0 seed read ofswas embedded in a backend node invisible to DFA, so DSE deleted the incomings := <start>store; the seed now goes through a plain DFA-visible scalar copy.testtool: fixed the run phase on unix targets (.exewas appended unconditionally) and injected-Facthreadson non-Windows so threaded-feature tests link the cthreads driver; 1200/1202 tests now pass on x86_64-linux.optloop.pas: satisfiedun_main's stricter case-exhaustiveness check after cherry-picking the optimizer work.-O4stage-2 self-compile of the compiler build clean: fixed DFA false positives and case-exhaustiveness warnings.Doublefor real-literal initializers; documented theRttiunit build for tests.compiler/msg/errore.msg;ppudumpupdated for the new option flags.fpcu.sh(invoke the in-treeppcx64with the built unit paths) andlazbuildu.sh(lazbuildusing the in-tree compiler) wrapper scripts; README Quick Start section on compiling programs and Lazarus projects with-O4.Testing
unleashed/tests/testfiles/(optlicm,optunswitch,optvect,optvrp,optvrpfold,optbitidiom,optbitscan,optsibcall,optstoremerge,optdistpat,optreassoc,optcasecluster,optcrossjump,optblockorder,optsink,optstoremotion,optswitchtable,optree,optshrinkwrap,optgvnpre,optrefelide,dead_store_ext,optpure,optpartialinline,optslp,optunrolldyn,optipara,opticf,stackalloc,optfinalvalue,sibcall,optsummary, and more), including must-fire, must-not-fire, safety, and disabled-switch cases, plus per-pass codegen assertion scripts (*_check.sh) proving at the assembly level that eligible shapes fire and rejected shapes do not.-O4with all new passes enabled (including full self-hosts with the opt-inSHRINKWRAPandGVNPREpasses forced on); the full unleashed suite passes with zero regressions beyond the documented pre-existing failures.Commits (oldest first)
ce247f6f99add -OoLICM loop-invariant code motion at -O4f85029776etests: optlicm loop-invariant code motion coverage5942317888add -OoLOOPUNSWITCH loop unswitching at -O4e3c1c8aca7add -OoBITIDIOM bit-population-count idiom recognition at -O4a146378dccadd -OoRANGEELIM value-range-analysis range-check elimination at -O4b58a765960add -OoVECTORIZE conservative SSE loop autovectorization87ef474c9eSigned mod by non-power-of-two constant: magic-number strength reduction612dd73ee3add tzcnt/bsr bit-scan idiom recognition to -OoBITIDIOM at -O4a417889ad1strength-reduce loopvar*invariant multiplies inside array-index/pointer offsetsb4892c6230add -OoVECTORIZE per-loop vectorization diagnostic (notes 06065/06066)68259ed54f-OoVECTORIZE: add scalar-broadcast and copy element-wise shapes9a57af019foptdeadstore: extend DSE to record-field and static-array-element stores6451f6addcaoptx86: sibling-call frame reuse for tail calls from a framed routine445fbd6c8aAdd -OoJUMPTHREAD jump threading / nested re-test elimination at -O4fd7296cf5bopttree: fix normalize() hoisting block-exprs out of control-flow bodies (-Oodeadstore)e203bc363fcompiler: make an -O4 stage-2 self-compile build clean (DFA false positives + case exhaustiveness)d67e6b8af7inline vars: infer Double for real-literal init; document Rtti unit build for tests (task 273)77a02770c5add -OoLOOPDISTPAT loop-distribution pattern idiom recognition at -O4babb3b5494add -OoLOOPPEEL full loop peeling of small constant-trip loops at -O4971e03dd61add -OoLOOPSPLIT loop splitting on an induction-variable crossover at -O49293ace3feadd -OoLOOPFUSE loop fusion of two adjacent same-space loops at -O4aedba022e2add -OoIFCONVERT loop if-conversion to packed maxps/minps at -O4324f28bea6add -OoREASSOC floating-point reduction reassociation at -O4 (fast-math gated)0e1bef5300add -OoUNROLLJAM outer-loop unroll-and-jam at -O453f858294cadd -OoPREDCOM predictive commoning of cross-iteration reloads at -O4455f28051cadd -OoSRA scalar replacement of non-escaping local aggregates at -O410b1ba8d85add -OoSTOREMERGE adjacent narrow-store coalescing at -O4a1f1438bbfadd -OoCASECLUSTER gcc-style case clustering at -O4a6978eadf3add -OoCROSSJUMP cross-jumping / tail merging at -O48e5adcad82add -OoBLOCKORDER static hot/cold basic-block layout at -O4776b611da0add -OoSINK partially-dead code sinking at -O4c156929359add -OoSTOREMOTION loop store motion / scalar promotion at -O48b4171b34cadd -OoVRP interval value-range propagation with branch folding at -O4ad7351332dadd -OoREFELIDE managed refcount elision (opt-in)a484c09620fix REASSOC miscompile/internalerror when the counter appears in the accumulated expression1cb1ab4e9badd -OoSWITCHTABLE case-to-lookup-table conversion at -O462aeea0f98add -OoREE redundant sign/zero-extension elimination at -O4f7f8d2c030add -OoSHRINKWRAP shrink-wrapping (opt-in), sinking a push-only prologue below a guard clausecf11e776f4add -OoGVNPRE dominator-based global value numbering / full-redundancy eliminationd4645bc067add Quick Start section to README: compiling programs and Lazarus projects with -O40e595fe7acadd fpc.sh wrapper: invoke in-tree ppcx64 with -O4 and built unit paths5f238a038dextend -OoVECTORIZE with single-precision sum / dot-product reduction vectorizatione45bbf4b59rename fpc.sh to fpcu.sh to avoid confusion with the fpc driver; document it in README Quick Startb1213a0f2adrop forced -O4 from fpcu.sh so debug builds stay debuggable; adjust READMEa7486cf488add lazbuildu.sh wrapper: lazbuild with the in-tree compiler via fpcu.sh14f577ffa7lazbuildu.sh: use the lazbuild on PATH instead of a sibling checkoutcfdeaf5297add -OoPURE interprocedural pure/const discovery, consumed by -OoLICM1de4a6465badd -OoPARTIALINLINE partial inlining / function splitting (procedures)118bcd0684add -OoSLP superword-level parallelism (SLP) straight-line vectorization247fba5de0add -OoUNROLLDYN dynamic-trip loop unrolling and -OoPREFETCH software prefetch98d932ba51extend -OoPARTIALINLINE partial inlining to functions (non-void return)e9e466506d-OoGVNPRE: consume the -OoPURE verdict (CSE identical pure calls); broaden pure eligibility to const-aggregate readersce7d807420add -OoICF identical code folding (opt-in), folding duplicate routine bodies via jump thunks4ea8de1765add -OoSTACKALLOC escape-analysis stack allocation of non-escaping local dynamic arrays95c78053ecadd -OoIPARA interprocedural register allocation (opt-in), narrowing caller-save spills to a callee's proven volatile clobber setfe0a4fbbe0add -OoFINALVALUE final value replacement + dead loop elimination (opt-in)62275fd46ashared cross-unit PPU optimizer summaries: cross-unit -OoPURE + -OoIPARA39786cb38c-OoICF: symbol-alias mode + cross-unit identical code foldingc2a48b0437-OoVECTORIZE: double-precision loop autovectorization (128-bit SSE/AVX)7ac3c112fe-OoPURE: -vh purity hints + read-only method/accessor eligibilitye620c046ccFix cs_opt_level2 const-fold spurious overflow on internal wraparound4bf0beabd2add -OoSIBCALL sibling-call optimization (opt-in)a7a109c1bf-OoFINALVALUE: pointer-stride accumulators + multi-statement bodies26da550200-OoPURE: treat pure/const calls as non-barriers in extended dead-store eliminationad3964f734optvectorize: fix -Oodeadstore x -OoVECTORIZE reduction miscompile (stale accumulator seed)f7be469997-OoFINALVALUE follow-ups: 64-bit counters, -Cr range-check regime, static/global accumulators5898a20945-OoSIBCALL: multi-location result forwarding + frame-address escape gate1f5e957d97fixup after cherry-picking main optimizer work: satisfy un_main's stricter case-exhaustiveness check in optloop.pasc95bce2901testtool: fix run phase on unix targets